Skip to content

fix(cli): every author-time rule that can gate runs on all three commands (#4409) - #4445

Merged
os-zhuang merged 3 commits into
mainfrom
claude/cli-lint-command-coverage-drift-w0nbcg
Aug 1, 2026
Merged

fix(cli): every author-time rule that can gate runs on all three commands (#4409)#4445
os-zhuang merged 3 commits into
mainfrom
claude/cli-lint-command-coverage-drift-w0nbcg

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #4409.

问题

os validate / os build / os lint 各自手工接线自己的规则子集。三份 import 列表之间没有任何东西相连,"这个命令跑哪些规则"只能靠肉眼 diff 三个 800 行的文件——每落一条规则答案就漂一次。26 条规则里 23 条只跑部分命令,其中 9 条能报 error

最危险的方向不是最显眼的那个:os build(负责发布的那个命令)是三个门里最弱的。一个审批人表达式解析不了的流(approval-expression-invalid)构建、发布一路绿灯,只有 os lint 拦得住,而 CI 通常跑的恰恰是另外两个。

这是同一失败模式的第五次出现(#3583#3782#4384/#4394#4402)。每次都修掉"那一例",留下模式:一条规则的命令覆盖 = 作者记得敲什么,忘了就是静默的。#4402 的守卫拦不住剩下的 23 条——它只过滤某一个 suite 的现有成员名,suite 之外被手工接进两个命令的规则,它一声不吭。

改成什么

把"记得"换成一张表。

packages/cli/src/lint/authoring-rules.ts — 26 条规则全部登记为数据:tier(gating / advisory)、读哪一层 stack(pre-parse normalized vs parsed)、跑哪些命令、以及唯一一处收窄的书面理由。三个命令都通过 runAuthoringRules() 消费,三个命令文件合计缩了约 1000 行。

守卫从"点名单条"升级为棘轮authoring-rule-wiring.test.ts,取代 reference-integrity-wiring.test.ts):

行为验证authoring-rule-command-parity.test.ts)——证明的是判决,不只是接线。一个命令可以跑了规则却不按它的结论 gate,#3782 那四条就是这么"接上了"的。每条原先有盲区的 gating 规则一个用例,外加 issue 自己的 repro 用真实 CLI 进程跑。

实测

issue 里的复现(app-todo 植入 CEL 坏掉的 expression approver):

之前 现在
os lint EXIT=1 ✗ EXIT=1 ✗
os validate EXIT=0 EXIT=1 ✗
os build EXIT=0(照常产出) EXIT=1 ✗

三个示例 app(todo / crm / showcase)× 三个命令仍全绿;告警量基本不变(lint 122→124,validate 4→4,build 2→5,新增的正是 build 原先瞎掉的那些)。

顺带修的同类洞

collectAndLintDocs 会让 os build 失败却从不在 os validate 跑——同一类"build 拒绝、validate 放行",只是方向相反。它逃过旧守卫是因为旧守卫按 lint*/validate* 命名约定匹配,而它叫 collect*。现在守卫逐个点名共享的非 registry gate,而不是靠模式匹配去撞。

两点行为变化

  1. 不再在第一个失败的 gate 就退出——一次跑完报全部错误。作者三个不相干的问题现在一次看全,而不是三个来回。副作用:--strict 现在覆盖所有 advisory,而不是原先恰好被 inline 打印的那一半。
  2. os lint 现在会因 gating 规则退出 1——这正是"预检不撒谎"的定义。

关于成本

成本从来不是维持现状的理由:typescript(~9MB)和 sucrase 已经是惰性加载(@objectstack/lintlazy-deps.test.ts 钉住了这个契约),而这组里最重的 validateReactPageProps#4340 起就以 suite 成员身份跑在三个命令上,没人察觉过开销。所以 gating 层没有一条需要按成本豁免。唯一收窄的 lintUniqueDeclarations 理由是 os lint 已经通过 lintDataModel 报同一条规则,登记进去会重复报告——这是记录下来的覆盖,不是缺失的覆盖。

注:issue 矩阵里 validateReactPageProps 那一行是旧的(统计于 27f907252)。它在 #4340 收尾时已进 suite,三命令都跑。

验证

  • CLI 665 tests、@objectstack/lint 729 tests 全过
  • tsc --noEmiteslint --no-inline-config 干净
  • 三个示例 app × 三个命令 EXIT=0

文档

content/docs/deployment/validating-metadata.mdx 的"one gate, two entry points"改成三入口的完整矩阵;cli.mdxos lint「beyond validate's hard gates」的说法此前是不准确的,已改为如实描述。


Generated by Claude Code

…ands (#4409)

`os validate`, `os build` and `os lint` each hand-wired their own subset of the
author-time rules. Nothing connected the three lists, so "which rules run here?"
was answerable only by diffing three 800-line files by eye — and the answer
drifted every time a rule landed. The audit found 23 of 26 rules running on some
strict subset, nine of them able to emit `error`.

The worst direction was the least obvious: `os build` — the command that
PUBLISHES — was the weakest gate of the three. A flow whose expression approver
does not parse (`approval-expression-invalid`) built and published green; only
`os lint` stopped it, and CI usually runs the other two.

This is the same failure mode's fifth appearance (#3583, #3782, #4384/#4394,
#4402). Each earlier repair removed an instance and left the MODE: a rule's
command coverage was whatever its author remembered to type, and forgetting was
silent. #4402's guard could not catch the rest — it filtered on the current
member names of one suite, so a rule hand-wired into two commands from outside
that suite passed it without a word.

Replace remembering with a table:

- `packages/cli/src/lint/authoring-rules.ts` declares all 26 rules as data —
  tier (gating/advisory), which stack tier they read (pre-parse vs parsed),
  which commands run them, and a written reason for the one narrowing. All three
  commands consume it through `runAuthoringRules()`; the three command files
  shrink by ~1000 lines between them.
- `authoring-rule-wiring.test.ts` upgrades the guard from a name list to a
  ratchet: a gating rule on fewer than three commands fails, a narrowed rule with
  no reason fails, a command that calls a rule directly fails, and an `advisory`
  claim is checked against the rule's own source so a gate cannot wear an
  advisory label to buy partial coverage. Remaining direct calls are listed with
  reasons in `DIRECT_CALL_RATCHET` / `LINT_IMPORT_RATCHET`.
- `authoring-rule-command-parity.test.ts` proves the verdict, not just the
  wiring: one case per previously-blind gating rule, plus the issue's own repro
  driven through the real CLI — exit 1 on all three commands where it was
  1/0/0 before.

Two things fall out of one report per run rather than exiting at the first
failing gate: an author with three unrelated problems sees all three, and
`--strict` now covers every advisory instead of the roughly half that happened
to be printed inline.

Also closes the same hole one gate over: `collectAndLintDocs` failed `os build`
and never ran on `os validate`, invisible because the parity guard keyed on the
`lint*`/`validate*` naming convention and that gate is called `collect*`. The
guard now names each shared non-registry gate explicitly.

Cost is not what argued against this: the heavy deps (typescript, sucrase) are
already lazy, and the heaviest rule of the set has run on all three commands as
a suite member since #4340 without anyone noticing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sz61GE1CWCSnGW4qShSEXs
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 1, 2026 7:54am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests labels Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/cli.

20 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/skills-reference.mdx (via packages/cli)
  • content/docs/api/client-sdk.mdx (via @objectstack/cli)
  • content/docs/api/data-flow.mdx (via @objectstack/cli)
  • content/docs/api/environment-routing.mdx (via @objectstack/cli)
  • content/docs/api/error-catalog.mdx (via @objectstack/cli)
  • content/docs/automation/hook-bodies.mdx (via packages/cli)
  • content/docs/deployment/backup-restore.mdx (via @objectstack/cli)
  • content/docs/deployment/cli.mdx (via @objectstack/cli)
  • content/docs/deployment/self-hosting.mdx (via @objectstack/cli)
  • content/docs/deployment/validating-metadata.mdx (via packages/cli)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/cli)
  • content/docs/kernel/runtime-services/data-service.mdx (via packages/cli)
  • content/docs/kernel/runtime-services/index.mdx (via packages/cli)
  • content/docs/permissions/authentication.mdx (via @objectstack/cli)
  • content/docs/plugins/index.mdx (via @objectstack/cli)
  • content/docs/plugins/packages.mdx (via @objectstack/cli)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/cli)
  • content/docs/protocol/kernel/realtime-protocol.mdx (via @objectstack/cli)
  • content/docs/releases/implementation-status.mdx (via @objectstack/cli)
  • content/docs/releases/v16.mdx (via @objectstack/cli)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added tooling and removed tooling labels Aug 1, 2026
…4409)

The role-word ratchet (ADR-0090 D3) counts occurrences per file, and two doc
lines describing `validateSemanticRoles` reintroduced the banned word for a
non-permission concept. Name what the rule actually checks instead:
`stageField` / `highlightFields` pointers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sz61GE1CWCSnGW4qShSEXs
@os-zhuang
os-zhuang merged commit 4bee182 into main Aug 1, 2026
18 checks passed
@os-zhuang
os-zhuang deleted the claude/cli-lint-command-coverage-drift-w0nbcg branch August 1, 2026 08:08
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 3, 2026
…4463) (objectstack-ai#4715)

* feat(metadata-protocol,lint): run the author-time rules at the runtime publish gate (objectstack-ai#4463)

The 26 author-time rules objectstack-ai#4409 collected behind one registry ran on `os validate`,
`os build` and `os lint` — three CLI commands. Every runtime metadata write (Studio,
REST /meta CRUD, MCP/AI authoring) reaches `saveMetaItem`, which did a per-type Zod
`safeParse` and stopped, so none of the 26 ran there. For a tenant that was not the
weakest of four doors but the only one: a sys_metadata overlay row is not in the
CLI's config file, so no command could have caught it.

Shape follows the ruling: one shared core, one runtime gate.

- Move AUTHORING_RULES (and the five rule modules that lived beside it) from
  packages/cli into @objectstack/lint; the CLI now calls the same table.
- Add the kernel-safe subpath entry @objectstack/lint/runtime, guarded by a new
  runtime-lazy-deps.test.ts: neither typescript nor sucrase loads at import or
  while gating. lazy-deps.test.ts is untouched.
- Declare `surfaces` + `runtimeTypes` / `surfaceReason` per rule, and extend the
  objectstack-ai#4445 ratchet to the new surface: a rule that answers neither fails, a gate that
  hand-wires a rule fails, a gate that does not run the core fails.
- Gate `state: 'active'` saves and the draft->active promotion for `flow` with the
  flow/approval/expression/reference families; 422 in the existing structured-issues
  envelope. Drafts pass (D1). Evaluation is differential, so only the submitted item
  can refuse its own write and stored rows are never re-judged (D4).
- Escape hatch OS_ALLOW_UNLINTED_METADATA_WRITES=1, loud on every use.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny

* fix(lint,objectql): escape the fingerprint separator; widen the publish repo double (objectstack-ai#4463)

Two CI failures on the first push, both real.

1. `check:nul-bytes` — the differential evaluator's finding fingerprint used
   three BARE NUL bytes as its separator. Runtime behaviour was fine; the file
   was not: grep/ripgrep classify a file containing a raw NUL as binary and
   return zero matches for it, silently, so the module disappears from code
   search and every grep-driven lint. git never warned because it sniffs only
   the first 8000 bytes and this NUL sat at byte 5361. Particularly bad here —
   the ratchet this PR adds is itself grep-driven, and the vanishing file was
   in the package it scans. Now the escaped U+0000 form: byte-identical at
   runtime, greppable at rest. (The shorter escape is deliberately avoided —
   a following digit turns it into a legacy octal escape.) Matches the
   existing convention at packages/rest/src/rest-server.ts:1065.

2. `Test Core` — objectql's publish double stubbed `getOverlayRepo` with
   `promoteDraft` alone, so `promoteDraftForPublish` reading the pending draft
   for the gate failed with `repo.get is not a function`. The real
   `SysMetadataRepository` has `get`; the double was narrower than the contract
   it stands in for (objectstack-ai#4550's shape), so the double is what is wrong. Widened
   it. The gate is NOT made conditional on the method existing — a repo-shaped
   capability sniff would silently reopen the `?mode=draft` + `POST /publish`
   bypass this PR closes, which is worse than the TypeError.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny

* fix(metadata-protocol): exempt `os migrate meta --stored` from the authoring gate (objectstack-ai#4463)

Found while auditing every internal `saveMetaItem` caller, not from a red
build — `migrateStoredMetadata` re-saves rows that ALREADY EXIST with
`mode:'publish'`, so the new gate refused any stored row carrying a violation.

That inverts the tool and contradicts D4. The migration's job is to rewrite a
stored body into the current dialect; it is not an author publishing anything.
Refusing meant a tenant holding one pre-existing bad flow could never
canonicalize that row: the migration reported `outcome:'failed'` and left the
body in the OLDER dialect — worse than the state it was asked to improve, and
precisely the "存量行走 ADR-0087 的老路" D4 promises.

Carve-out is keyed on `source === 'migrate-stored'`, which is stated by the
SERVER and never forwarded from a request (see the provenance note at the top
of `saveMetaItem`), so no caller can spell its way past the gate — pinned by a
test that walks four near-miss spellings and expects 422 for each.

`duplicatePackage` is deliberately NOT exempt: it mints brand-new rows under
new names, and a copy of a broken flow is a new broken flow. Its per-row
failure reporting already surfaces that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

2 participants